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REMARKS/ARGUMENTS 

This Amendment and Response is promptly filed to place the above-referenced case in 
condition for immediate allowance. 

The status of the claims is as follows: 

Cancelled: 1 - 6, 37 - 42, 45, 46, and 47; 

Amended: 7, 17, 20, 25, 34, 43, and 44; 

Added: None; and 

Currently outstanding: 7 - 24, 25 - 36, 43, and 44. 
No new matter has been added to the application. 

From the outstanding Office action, the Examiner rejected Claims 1-47 on a variety of 
bases including Sections 101, 102 and 103. 

Applicants have amended their Claims to better set forth the subject matter of the 
present technology but reserve all right to pursue additional disclosure in the future. 

Reconsideration is respectfully requested. 

No narrowing amendment to conform with statute has been made in the Application by 
amendments to the claims, but instead the amendments are directed to further clarification of 
Applicants' technology. 

In the outstanding Office Action, the Examiner indicated that certain items were 
missing from the Information Disclosure Statement filed September 5, 2003 (believed to be 
actually December 3, 2003). Enclosed herewith separately are copies of articles AE and AN 
from page 1 and AH from page 3. Article AC from page 1 is not yet available but it will be 
submitted as soon as made available. 
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The Examiner made certain claim objections to Claims 3, 17, and 43. These objections 
are believed to be resolved by the amendments to the claims. 

The Examiner rejected certain of the claims under 35 USC § 101 as the claimed 
invention was considered by the Examiner to be directed to non-statutory subject matter. The 
amendments to the claims are believed to resolve this rejection by further clarification of 
Applicants' claimed subject matter. It is believed that the State Street case (149 F.3d 1368, 47 
USPQ 2 nd 1596 (Fed.Cir.)) is applicable. Applicants particularly point out the utility of being 
able to provide date in a compressed form so that it occupies less storage space in a storage 
medium, electronic or otherwise. Furthermore, tangibility of Applicants' claimed invention is 
present in the testability, detectability, reliability, and controllability of the resulting 
compressed data as stored in a storage system. 

The Examiner rejected several claims under 35 USC §112, first paragraph. The 
amendments to the claims are believed to resole these rejections. 

Applicants have removed the term "3-dimensional video" from their claims. The term 
"energy" is generally known in the art with regards to intensity or other optical characteristics 
of a pixel, picture element, or image element. The term "intensity" is generally well-known in 
the art as is reflected by the Bright '977 patent. Applicants would generally be willing to 
revise their application to conform it with the vocabulary that is known in the art should it be 
deemed necessary. 

The phrase "other learning regimes" has been removed from the claims. 

Due to the foregoing, the rejections based on 35 USC §112, first paragraph are 
believed to be overcome. 
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The Examiner also rejected the claims based on 35 USC §102 and §103. Basically, for 
the claims as amended, the Bright '977 patent (Bright) is the basis for the Examiner's 
remaining objections. 

Applicants believe that the Examiner has misinterpreted the Bright '977 patent and its 
relevance to the Yadegar et al compression method. The stark and fundamental differences 
between the Yadegar et al compression method and that due to the Bright '977 patent can be 
enumerated as below: 

1. Yadegar et al apply tri-partite adaptive filtering method whereby; 

1.1 Filter 1 treats the quasi-uniform regions in the image using on-line regression 
learning methods. In contrast, the Bright '977 patent offers an experimentally and theoretically 
verified inferior data-drive (non-learnable) method of linear interpolation method. 

1.2 Filter 2 treats the piece wise extended and organized structures in the image 
using on-line and off-line adaptive non-linear regression learning methods. In contrast, the 
Bright '977 patent does not at all offer any method for treating piecewise extended organized 
structures. 

1.3 Filter 3 treats the texture structures in the image using on-line and off-line 
adaptive non-linear regression learning methods to efficiently compress texture pattern. In 
contrast, the Bright '977 patent only suggests in brief passing a template matching method that 
requires large database of texture patterns and an extremely efficient search mechanism for 
texture pattern matching none of which are at all discussed in the Bright '977 patent. 

2. Yadegar et al apply lossless adaptive on-line preconditioning/learning arithmetic 
coding to the residuals from Filter 1, Filter 2, and Filter 3 each separately for optimal last drop 
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compression. In contrast, in the Bright '977 patent there is no mention of lossless coding to 
the residuals. 

Based on the above comparative analysis, the Bright patent can not act as disabling 
prior art to Applicants' claims. Applicants' claims go beyond that which is disclosed in the 
Bright '977 patent. 

Furthermore, the addition of the Tsishkou et al. reference ("Tsishkou") does nothing to 
resolve the shortcomings of the Bright patent. As a result, all of Applicants' claims as 
amended are patentable over the Bright patent either alone or taken together with any 
reasonable combination of the cited references. Applicants believe that this extends also to any 
of the cited references taken in any reasonable combination in that none of them disclose 
Applicants' subject matter as claimed. 

In order to better indicate their system, Applicants provide some additional descriptive 
material with regards to their system below. 

Applicant' first filter decomposes the image into triangular tiles such as Peano-Cesaro 
tiling and applies on-line machine learning in the form of regression to each tile for modeling. 
The intention behind this stage of the algorithm is to capture quasi-uniform regions in the 
image using regression techniques. The algorithm appiles optimization methods to learn the 
most optimal model for intensity distribution over a tile in order for the approximation error to 
be below an error threshold. 

The input to the regressor is not only the intensities at the vertices of the tile but also 
some intensities from neighboring tiles that are already modeled. This last extra information 
input to the regressor from neighboring tiles increases the likelihood of successful modeling. 
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If the tile is successfully modeled by Filter 1 : 

Declare tile terminal, which means do not consider 
it for further processing. 
Go to the next tile. 
Else if tile is not successfully modeled by Filter 1: 
Decompose tile into subtiles. 
Put subtiles in a priority queue data structure to be 
modeled later in the process. 
Go to the next tile. 

Applicants have conducted extensive experimentation on linear interpolation, linear 
regression and non-linear regression methods. Experiments strongly favor regression methods 
over liner interpolation, though for quasi-uniform regions the enhancements obtained by the 
application of non-linear regression over linear regression is small. For this reason Applicants 
have abandoned applying linear interpolation, which is the technique adopted by Bright, and 
have selected linear regression method over non-linear regression methods because of its 
computational efficiency. It is to be noted that Applicants have evidence from people working 
with them as well as documented evidence that their linear interpolation methods go back to the 
early 1990's. 

By the time the algorithm is done with Filter 1, most of the quasi-uniform regions in 
the image frame have been captured and modeled. The remaining triangular tiles are medium 
range in size (about 17x17 pixels) and contain either (i) organized piecewise extended 
structures or (ii) texture, or (iii) combination of organized structures and texture. The 
intension behind this stage of Applicants' algorithm is for Filter 2 to model the organized 
piecewise extended structures in the image. Therefore, by the end of Filter 2 process, the 
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algorithm is left with texture, which may be interpreted as irratic and noisy structures. The 
texture region is then treated by the Filter 3 process. 

For the Filter 2 process, Applicants have conducted extensive research and 
development applying adaptive structure modeling and machine learning methods to 
successfully model organized peicewise extended structures. The machine learning methods 
that Applicants have worked with, such as a mixture of Guassians, neural nets, Support Vector 
Machine (SVM), and Principle Component Analysis (PC A), can be considered as non-linear 
regression methods that have considerable learning power to learn and successfully model 
organized structures in medium size tiles. 
Figure 6 exhibits some samples of piesewise extended structures. 




Figure 6 



Filter 2 determines and traces the zero-crossing gradient regions in the tile. The blue 
traces in figure 6 are where zero-crossings take place. 

The next step is to model the intensity distributions on either side of the zero-crossing 
traces bounded by a narrow strip of order 9 pixels in width using either on-line adaptive 
diffusion methods or off-line machine learning methods, either of which are highly non-linear 
regression by nature. 
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The final step in Filter 2 is to model the recessing, more or less quasi-uniform regions 
in the tile, which again is handled by either on-line adaptive or off-line machine learning 
methods. 

If tile is successfully modeled by Filter 2: 

Declare tile terminal, which means do not 
consider it for further processing. 
Go to the next tile. 
Else if tile is not successfully modeled by Filter 2: 
Decompose tile into sub-tiles. 
Put sub-tiles in a priority queue data structure to be 
modeled later in the process. 
Go to the next tile. 

The priority queue data structure used in Filter 2 inherently contains a sophisticated 
piece of strategy knowledge hardwired into its organization and processing. The tiles stored in 
the priority queue are ordered according to their likelihood of being successfully modeled if 
processed by Filter 2 mechanism. A tile has higher probability of success if a greater number 
of its neighboring tiles have already been successfully modeled, bearing in mind that a 
neighboring tile that has been modeled by Filter 2 mechanism offers more useful information 
due to the presence of piecewise extended structures in it than a neighboring tile modeled by 
Filter 1 mechanism due to the quasi-uniform distribution of intensities. Applicants have 
developed rigorous and extremely efficient algorithms to retrieve the complete set of 
neighboring tiles of a tile, which is a required step for Filter 2 processing. In the case of the 
Peano-Cesaro tiling in Figures 1 and 2, Applicants exploit the Peano-Cesaro sweep to establish 
a binary "state -labeling" or "code sequence" for the tiles as tile decomposition is applied. The 
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binary code sequence of a tile embeds an inheritance and relational knowledge about the tile to 
its parent, grandparents, ancestors, children, grandchildren, deeper generations of tiles, sibling 
and side and vertex adjacent tiles. Applicants pervasively exploit the binary code sequence in 
tile retrieval algorithms. 

By the time the algorithm is done with Filter 2, most of the piecewise extended 
structures in the image frame have been captured and modeled. The remaining triangular tiles 
are small size (about 5x5 pixels or less) predominately containing texture patterns. The 
intention behind this stage of Applicants' algorithm is for Filter 3 to model texture as an 
extension of the non-linier filtration of Filter 2. 

Therefore, by the end of Filter 3 process the algorithm is left to treat the residual 
information from Filter 1, Filter 2, and Filter 3 for the last drop compression using adaptive 
arithmetic coding. 

If the piecewise "extended" structures are made "less and less extended" in their 
structural patterns, they become less and less differentiable and hence more and more noise 
like. For instance, consider an edge running for a good length of say 20 pixels, before 
changing direction to another lengthy edge as illustrated in Figure 7. Such a structure is 
piecewise extended and can be successfully modeled by Filter 2 mechanism in Stage 3 of 
Applicants' algorithm. Figure 8 illustrates the same structure as in Figure 7 but now the 
lengths of the subtended edges are much smaller, say about 3 pixels in length. Figure 9 
depicts several corner structures such as those in Figure 8 entangled and crisscrossing each 
other in a region of the image. This entangled texture organization of intensity distribution as 
shown in Figure 9 is no longer piecewise extended structure and can no longer be handled by 
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Filter 2 mechanism. It is the responsibility of Filter 3 to model texture behavior such as in 
Figure 9. 






Figure 7 Figure 8 Figure 9 

The analysis above on how in the limit piecewise extended structure become noise like 
and texture lead to the following important conclusion. Piecewise extended structures such as 
in Figure 7 preserve local invariance for a good extend length. Thus an edge in Figure 7 
preserves the same gradient along the edge for some 20 pixels. This local invariance is of key 
significance to Filter 2 mechanism and the successful modeling of piecewise extended 
structures. 

In stark contrast, texture behavior does not exhibit local invariance for, as Figure 9 
illustrates, the gradient pattern keeps rapidly changing from one to the other. However, 
texture patterns to lesser or greater degree, depending on the particular texture, exhibit global 
correlation in their patterns. Thus in Figure 9 if the pattern of the crisscrosses spread over a 
region in the image, the global texture pattern reveals a global similarity that can be exploited 
for compression purposes and this is precisely what Applicants' Filter 3 does. 

Applicants have done and continue to do extensive research and development in 
maturing and optimizing various algorithms for texture modeling. According to Yadegar et al, 
texture template matching is a poor and elementary method of modeling texture. Texture 
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template matching would require a large database of texture patterns to be kept in memory as 
part of the codec and it also requires an efficient real-time split-second search mechanism for 
optimal template matching. Applicants' generally do not follow texture matching. 

Applicants have conducted and continue to conduct machine learning methods such as a 
mixture of Gaussians, inpainting methods and random field methods for texture modeling. All 
these methods at their core are adaptive and non-linear and they all attempt to off-line or on- 
line learn or adapt texture patterns at various localities in the image and fill in for (or inpaint) 
wider texture areas in the image using the learned or adapted texture samplings from various 
localities. 

It is clearly the case that due to less correlation in information in a texture based region, 
the compression factors obtained in Filter 3 are less than those obtained in Filter 2. Indeed a 
statement can be made that the compression factors obtained from Filter 1 are greater than 
those obtained from Filter 2, which in turn is greater than those obtained from Filter 3. 

The important point to bear in mind is that these compression factors from Filter 1, 
Filter 2 and Filter 3 have multiplicative (and not additive) effect on each other. Thus, for 
instance, compression factors of 15 (from Filter 1), and 3 (from Filter 2) and 1.3 (from Filter 
3) will yield a global compression of order 15 X 3 X 1.3 » 58 (and not 15 + 3 +1.3 « 19). 

If tile is successfully modeled by Filter 3: 

Declare tile terminal, which means do not consider 
it for further processing. 
Go to the next tile 

Else if tile is not successfully modeled by Filter 3: 
Decompose tile into sub-tiles. 
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Put sub-tiles in priority queue data structure to be 
modeled later in the process. 
Go to the next tile. 

Applicants subject the residual information from Filter 1, Filter 2 and Filter 3 to 
adaptive arithmetic coding for last drop compression. However, before subjecting the residual 
information from Filter 2 to adaptive arithmetic coding, Applicants have observed that this 
residual information still maintains some consistency and correlation in piecewise extended 
structural forms. Experiments conducted have revealed that off-line or on-line machine 
learning or adaptive modeling can be further applied to this residual information by Filter 2 to 
further gain compression. As a result of this finding, the "first order residual" from Filter 2 
goes into a second round of Filter 2 processing for further compression. The output of this 
second round of Filter 2 processing is the "second order residual" which is next subjected to 
adaptive arithmetic coding. 

Adaptive arithmetic coding is an online learning/preconditioning mechanism for lossless 
coding. When this adaptive arithmetic lossless coding first starts the last drop compression of 
the residual information, it has no knowledge about the entropy of the residual information. 
As it penetrates into the stream of the residual information, it builds a probability distribution 
model of the stream that dynamically updates itself as the steam of residual information is 
processed. The probability model at each instance in the process assigns least code to the 
highest recurring residual pattern and follows that ranking throughout the processed portion of 
the residual stream. As mentioned, this probability model gets updated as the lossless 
arithmetic coding advances into the residual stream. By the time the process is complete and 
the residual stream is exhausted, the most optimal code assignment has been preconditioned. 
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Extensive experiments conducted by Applicants indicate that higher compression factors 
are achieved by keeping residuals from Filter 1, Filter 2 and Filter 3 separate. This is a 
natural conclusion, since Filter 1 deals with quasi-uniform regions and the residual from Filter 
1 shows greater entropy consistency (that is, lower entropy) by keeping it separate than having 
it mixed with the residual from Filter 2, which deals with piecewise extended structures. 
Similar arguments apply to residuals from Filter 2 and Filter 3. 

The Tsishkou et al publication describes an image compression algorithm that is 
suitable for a niche domain specific class of images such as ultrasound images of the heart. 
Therefore, the compression method is highly class based and it can not be exploited in a setting 
with generic images. 

The core of the Tsishkou et al algorithm is composed of two parts: (i) the hierarchical 
database generation of templates of visual descriptors using a large training set of cardio 
ultrasound images and (ii) devising a two stage search mechanism for template matching to 
find the closest match in the database for run-time cardio ultrasound image compression 
application. 

The Tsishkou et al algorithm shows wide divergence from Applicants' algorithm. 
Tsishkou et al algorithm is strongly class based whereas Applicants' algorithm can be made to 
be generic or class based depending on the specific application. Tsishkou et al algorithm 
requires off-line generation of a hierarchical database of indices whereas Applicants' algorithm 
requires on-line and off-line linear and non-linear regression machine learning regime whereby 
the internal parameters of the learning regime are tuned on the basis of the training set - this 
difference makes Applicants' method far more scalable. It is important to note that Applicants' 
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algorithm requires no database of templates of any sort, none whatsoever. Tsishkou et al 
algorithm requires construction of an explicit, extremely efficient search algorithm for template 
matching whereas Applicants' machine learning paradigm totally overcomes the search hurdle 
since, during training and adjustment of the internal parameter of the learning regime, a 
mapping is established between the training set and the output of the learning regime so that at 
run-time this mapping is spontaneously activated for a match. Tsishkou et al do not spell out 
any specific decomposition scheme: Whether triangular or rectangular? Binary, quaternary, 
ternary or what? The caption of Figure 1 in Tsishkou et al publication states segmentation 
based on polar coordinates. Yadegar et al's algorithms are not specific to polar or Cartesian 
coordinate systems and they have been applied to a variety of triangular decomposition 
techniques. Tsishkou et al do not spell out the specific lossless coding method other than 
stating that they apply entropy coding. Applicants have applied a variety of lossless coding 
methods such as differential coding, run length coding, Huffman coding, arithmetic coding and 
adaptive arithmetic coding. Applicants' experiments demonstrate adaptive arithmetic coding to 
be superior. 

The Examiner has also cited a number of patents and publications as pertinent to the 
presently claimed invention. Since none of these have been relied upon as a reference against 
Applicants' claims, no further comment is deemed necessary. 

In view of the above, the Examiner is respectfully requested to reconsider his position 
in view of the remarks made herein and the structural distinctions now set forth. The 
Examiner's rejections of the outstanding claims are believed to no longer apply. It is now 
believed that this application has been placed in condition for allowance, and such action is 
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respectfully requested. Prompt and favorable action on the merits is earnestly solicited. 
Applicants respectfully request that a timely Notice of Allowance be issued in this case. 

The statements made herein with respect to the disclosures in the cited references 
represent the present opinions of the undersigned attorney. In the event that the Examiner 
disagrees with any of such opinions, it is respectfully requested that the Examiner specifically 
indicate those portions of the respective references providing the basis for a contrary view. 

If the Examiner believes that a telephone or other conference would be of value in 
expediting the prosecution of the present application, enabling an Examiner's amendment or 
other meaningful discussion of the case, Applicants invite the Examiner to contact Applicants' 
representative at the number listed below. 

With the above-referenced changes, it is believed that the application is in a condition 
for allowance; and Applicants respectfully request the Examiner to pass the application on to 
allowance. It is not believed that any additional fees are due; however, in the event any 
additional fees are due, the Examiner is authorized to charge Applicants' Attorney's Deposit 
Account No. 03-2030. 



Respectfully submitted, 
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